New Branching Rules Induced by Plethysm * B. Fauser

We derive group branching laws for formal characters of subgroups Hπ of GL(n) leaving invariant an arbitrary tensor T π of Young symmetry type π where π is an integer partition. The branchings GL(n) ↓ GL(n − 1) , GL(n) ↓ O(n) and GL(2n) ↓ Sp(2n) fixing a vector vi , a symmetric tensor gij = gji and an antisymmetric tensor fij = −fji , respectively, are obtained as special cases. All new branchings are governed by Schur function series obtained from plethysms of the Schur function sπ ≡ {π} by the basic M series of complete symmetric functions and the L = M −1 series of elementary symmetric functions. Our main technical tool is that of Hopf algebras, and our main result is the derivation of a coproduct for any Schur function series obtained by plethysm from another such series. Therefrom one easily obtains π-generalized Newell-Littlewood formulae, and the algebra of the formal group characters of these subgroups is established. Concrete examples and extensive tabulations are displayed for H 1 3 , H21 , and H3 , showing their involved and nontrivial representation theory. The nature of the subgroups is shown to be in general affine, and in some instances non reductive. We discuss the complexity of the coproduct formula and give a graphical notation to cope with it. We also discuss the way in which the group branching laws can be reinterpreted as twisted structures deformed by highly nontrivial 2-cocycles. The algebra of subgroup characters is identified as a cliffordization of the algebra of symmetric functions for GL(n) formal characters. Modification rules are beyond the scope of the present paper, but are briefly discussed.